MARIOnET: motion acquisition for robots through iterative online evaluative training

As robots become more commonplace, the tools to facilitate knowledge transfer from human to robot will be vital, especially for non-technical users. While some ongoing work considers the role of human reinforcement in intelligent algorithms, the burden of learning is often placed solely on the computer [2]. These approaches neglect the expressive capabilities of humans, especially regarding our ability to quickly refine motor skills. Thus, when designing autonomous robots that interact with humans, not only is it important to leverage machine learning, but it is also very useful to have the tools in place to facilitate the transfer of knowledge between man and machine. We introduce such a tool for enabling a human to transfer motion learning capabilities to a robot. In this paper, we propose a general framework for Motion Acquisition in Robots through Iterative Online Evaluative Training (MARIOnET ). Specifically, MARIOnET represents a direct and real-time interface between a human in a motion-capture suit and a robot, with a training process that provides a convenient human interface and requires no technical knowledge. In our framework, the learning happens exclusively by the human not the robot. However, the robot provides a natural interface for interaction, and is able to store and reuse trained behaviors autonomously in the future. Our approach exploits the ability at which humans are able to learn and refine fine-motor skills [6, 4]. Implemented on two robots (one quadruped and one biped), our results indicate that both technical and non-technical users are able to harness MARIOnET to quickly improve a robot’s performance of a task requiring fine-motor skills. Cite as: MARIOnET: Motion Acquisition for Robots through Iterative Online Evaluative Training (Extended Abstract), Adam Setapen, Michael Quinlan, and Peter Stone, Proc. of 9th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS 2010), van der Hoek, Kaminka, Lespérance, Luck and Sen (eds.), May, 10–14, 2010, Toronto, Canada, pp. XXX-XXX. Copyright c © 2010, International Foundation for Autonomous Agents and Multiagent Systems (www.ifaamas.org). All rights reserved. 1. MOTIVATION Historically robot motion has been written by experts, where behaviors are coded by hand or via extensive learning experiments using constrained parameterizations, causing a lot of wear and tear on the robots [3]. Generally, programming specialized robot motions requires a significant amount of coding, which is not possible for most people. We aim to develop an efficient method for generating cyclical open-loop sequences like stable robot gaits, without requiring expert knowledge or machine learning. Recent breakthroughs in behavioral motor control have enhanced our understanding of the human brain and illustrate how remarkable our innate capacity for delicate motor control is [6]. Muellbacher et al. report that given a 60minute training period, subjects can rapidly optimize performance of a complex task involving fine motor control [4]. We hope to harness this ability in this work. The high-level motivation for MARIOnET is that a realtime mapping from a human to a robot will serve as a convenient interface for quickly and systematically training efficient motion sequences. While there is certainly a difference in the dynamics of robots and humans, we believe that people’s ability to quickly hone fine motor skills can be exploited to rapidly train diverse robot motions. Even if the mapping from human coordinates to robot coordinates is not exact, we hypothesize that humans will be able to learn to correct for any inconsistencies. Additionally, the prospect of mapping any human limb to any robot limb allows for a flexible training process (e.g., mapping human arms to robot legs).